Understanding generalization error of SGD in nonconvex optimization

The success of deep learning has led to a rising interest in the generalization property stochastic gradient descent (SGD) method, and stability is one popular approach study it. Existing bounds based on do not incorporate interplay between optimization SGD underlying data distribution, hence cannot even capture effect randomized labels performance. In this paper, we establish error for by characterizing corresponding terms on-average variance gradients. Such characterizations lead improved experimentally explain random We also regularized risk minimization problem with strongly convex regularizers, obtain proximal SGD.